Relevance & Research Question: In 2014, about 22% of German Internet users accessed the Internet with mobile devices on a daily basis (see van Eimeren & Frees, 2014). However, small screens on smart phones challenge questionnaire designers. Mainly grid questions on mobile devices seem to increase response burden. If all items are to be presented on a single screen, small font sizes and small response options are required. Or, if not all response options are visible on load, additional scrolling is needed which might bias ratings (see Couper, Tourangeau, & Conrad, 2004). To determine the impact of questionnaire customization on data quality, we tested a responsive design that adapts the questionnaire depending on the device used for participation.

Methods & Data: In a split ballot design N = 3929 respondents were recruited in five online panels. They were either directed to a classic design questionnaire (i.e., the same layout for all devices, not optimized for touch screens, and grids on a single page) or to a responsive design questionnaire (i.e., layout depending on available screen size, optimized for touch screens, and grid questions split into single questions). Two grids and two quality check items (e.g., “click the second response option“) were included. Analyses are based on N = 428 (10.9%) respondents who used a smart phone to access the survey.

Results: We found less straight lining in the responsive design (8.6% vs. 16.8% in the classic design), which is an indicator of a beneficial effect on data quality. Completion with the responsive design took some longer (M = 18.9 vs. 17.0 minutes). This, however, seems to be due to deeper question processing as items for quality checks were more frequently answered correctly (90% vs. 78%). Furthermore, respondents gave better evaluations of the survey. No considerable differences were observed, though, regarding item nonresponse, the number of characters in open questions, and the distribution of ratings (e.g., means and top-2 boxes).

Added Value: The results provide further evidence that the questionnaire design should preferably adapt to the type of device respondents use to participate. A responsive design meets mobile participants’ needs and expectations and helps improve data quality.